Application-independent text entry for touch-sensitive display

ABSTRACT

A user interface method and apparatus for an electronic device operates by detecting ( 108 ) a stroke of a touch sensitive display ( 170 ) forming a part of the electronic device. The stroke is categorized ( 116 ) as one of a swish type stroke and a non-swish type stroke. If the stroke is a non-swish stroke, it is translated ( 132 ) into an application function. If the stroke is a swish stroke, it is converted ( 128 ) to a character input function. The touch sensitive display ( 170 ) has a grid ( 50 ) containing cells ( 51, 53 , etc.), with each cell containing a plurality of characters, and the grid can overlay application interface to thereby occupying a common area, up to and including the entire area, of the touch sensitive display ( 170 ).

FIELD OF THE INVENTION

This invention relates generally to the field of human interfaces forelectronic devices. More particularly, certain embodiments consistentwith this invention relate to an application-independent text entrysystem and method for a touch sensitive medium such as a touch sensitivedisplay.

BACKGROUND OF THE INVENTION

A desktop computing device usually offers two basic input devices: thekeyboard and the mouse. Text input is provided through the keyboard,while pointing (moving pointer, selecting) as well as most windowmanagement actions (resizing, scrolling, menu selection, etc.) areexecuted with the mouse. There is also some redundancy, since thekeyboard can also control navigation (arrow keys) and some windowmanager actions (with shortcut keys).

On a handheld device a touch sensitive area on the display is commonlyused as the only or primary input interface. The total display area onsuch a device may be shared for display, text entry and applicationicons. The size of such a display limits the user's ability to easilyenter data and limits the ability to have an easily usable dedicatedarea for text input. On such devices that use a dedicated input area,it's size may be too small to be accurately accessed by finger. In suchcases, a stylus is used to enter text.

BRIEF DESCRIPTION OF THE DRAWINGS

The features of the invention believed to be novel are set forth withparticularity in the appended claims. The invention itself however, bothas to organization and method of operation, together with objects andadvantages thereof, may be best understood by reference to the followingdetailed description of the invention, which describes certain exemplaryembodiments of the invention, taken in conjunction with the accompanyingdrawings in which:

FIG. 1 is an illustration of a swish movement consistent with certainembodiments of the present invention;

FIG. 2 is a table illustrating the characteristics of strokes on a touchsensitive display with a swish stroke contrasted with other types ofstrokes;

FIG. 3 illustrates an exemplary text entry grid layout consistent withcertain embodiments of the present invention;

FIG. 4 illustrates several swish strokes consistent with certainembodiments of the present invention;

FIG. 5 is an exemplary hand-held device such as a wireless telephone orPDA using a touch sensitive display consistent with certain embodimentsof the present invention;

FIG. 6 is a flow chart depicting operation of an electronic device suchas a hand-held PDA, messaging or paging device, or wireless telephone ina manner consistent with certain embodiments of the present invention;and

FIG. 7 is a block diagram of an exemplary electronic device that canutilize the data entry arrangement consistent with certain embodimentsof the present invention.

DETAILED DESCRIPTION OF THE INVENTION

While this invention is susceptible of embodiment in many differentforms, there is shown in the drawings and will herein be described indetail specific embodiments, with the understanding that the presentdisclosure is to be considered as an example of the principles of theinvention and not intended to limit the invention to the specificembodiments shown and described. In the description below, likereference numerals are used to describe the same, similar orcorresponding elements in the several views of the drawings.

The terms “a” or “an”, as used herein, are defined as one or more thanone. The term “plurality”, as used herein, is defined as two or morethan two. The term “another”, as used herein, is defined as at least asecond or more. The terms “including” and/or “having”, as used herein,are defined as comprising (i.e., open language). The term “coupled”, asused herein, is defined as connected, although not necessarily directly,and not necessarily mechanically. The term “program”, as used herein, isdefined as a sequence of instructions designed for execution on acomputer system. A “program”, or “computer program”, may include asubroutine, a function, a procedure, an object method, an objectimplementation, in an executable application, an applet, a servlet, asource code, an object code, a shared library/dynamic load libraryand/or other sequence of instructions designed for execution on acomputer system.

The terms “text”, “symbol” and “character” as used herein is intended toinclude characters and common punctuation characters along with anyother characters that might be desirably entered via a single stroke ofa keyboard (e.g., +, /, \, |, #, @, ˜, backspace, etc.). Additionally,these terms are to be considered both singular and plural, in that areference to entering a character can include making a single entry thatcontains multiple characters (e.g., commonly used combinations ofcharacters such as “QU” or “SH” may be handled in some data entryscenarios the same as if the combinations were a single character).Moreover, the terms “text”, “symbol” or “character” may take onequivalent meanings for character sets other than those commonly used inconnection with the English language.

The term “grid” as used herein can mean a conventional linear X-Y gridthat divides an area into multiple rectangular cells. However, the termcan also mean circular grids, spiral grids, irregular shaped grids, orother shapes than simple X-Y grid systems, with subdivisions of suchgrid systems also being referred to as “cells”, “zones” or “regions”.The term “grid” also encompasses the characters and/or commandsassociated with the grid. Depending on the context, the terms “zone” and“region” can also be used within a less restrictive dictionary meaningto identify a particular general area of a touch sensitive medium.

The text input user interface for small hand-held devices has alwaysbeen a data entry bottleneck, and the problem worsens as such devicesget smaller and increasingly powerful. A broad range of solutions havebeen proposed to tackle this issue, from foldable keyboards, to voicerecognition. Certain embodiments consistent with the present invention,relate to methods and apparatus for permitting the data entry area of atouch sensitive medium such as a touch sensitive display (ortouchscreen) to be shared with an application's display functions (e.g.,prompts, icons, data entry box, windows, menus, and other visualobjects, etc.) without conflict. If the text input area is shared withareas that display the application, conflicts may occur wherein thedevice of interest may receive input that could be interpreted either astext entry or application commands. Certain embodiments consistent withthe present invention seek to resolve such conflicts by segregatingclasses of data entry motions that correspond to text entry from thosewhich might represent selection of icons, menu items or otherapplication functions.

A text input method consistent with certain embodiments of the presentinvention is provided herein that is especially useful for a smallhandheld device with a touch sensitive display. Such embodiments can besimple, fast, accurate, and can avoid conflict with applicationfunctions. Certain embodiments consistent with the present inventionutilize a text entry motion referred to as a swish, which is illustratedin FIG. 1. A finger 20 touches the touch sensitive display surface 24 atpoint a, then slides approximately linearly to point b where it isreleased. A swish is, thus, a touch-move-release sequence of thetouching device, that A) has a non-negligible diameter; B) is locationdependent; C) has a negligible delay between the touch and the beginningof the move; and that ideally has the following characteristics: D) fast(short time lag between touch and release—e.g., approximately on theorder of 100 ms); E) small (small diameter); and F) simple (singledirection).

While one of the conditions D, E, or F could be relaxed as will bediscussed below, the first three conditions provide the definition of abasic swish as used herein. In alternative embodiments, other generalrequirements for defining a swish or an analogous or functionallyequivalent motion could be defined, without departing from certainembodiments consistent with the present invention. A sequence thatfulfills B and C but violates A is considered a tap. A sequence that islocation independent is considered a gesture. A sequence violating C butsatisfying A and B is either a selection (like when a word is selectedin a text processor) or a select and drag sequence (moving an objectfrom one place to another). A long swish may either retian the timet_(b)−t_(a) constraint or relax it according to the actual distancetraversed by the stroke. FIG. 2 can be used to clarify this definitionby illustrating different strokes that are or are not swishes, as wellas variations in acceptable swish type strokes when certain conditionsare relaxed. In FIG. 2, a stroke is presented as it would appear on thetouch sensitive display, along with a graph of curvilinear length versustime, in order to illustrate the time component.

With reference to FIG. 2, the time t_(b)−t_(a) for a swish will varyfrom user to user and will vary with the size and layout of the inputtouch sensitive medium. In experiments conducted with a palmtop computersize device, the average time t_(b)−t_(a) for a swish was approximately100 ms with times varying in a bell shaped curve ranging from about 30ms to about 170 ms, with a few times being as long as 500 ms. Anappropriate range of times that can define an appropriate time range orthreshold to define a swish can be determined experimentally for a giventouch sensitive medium's layout and size. The average distance used toproduce a swish key stroke in the above experiments was approximately0.75 times the height or width of a square key, and ranged fromapproximately 0.1 to approximately 2 times the height or width of asquare key. These experimental characteristics can be used as a startingpoint for establishing threshold boundaries that can be used to make adecision as to whether or not a swish type stroke has been executed, butfurther refinement is possible for any given touch sensitive medium andkey layout. Thus, these time and distance parameters should beconsidered a guideline for the start of experiments and should not beconsidered limiting.

In accordance with certain embodiments consistent with the presentinvention, a text or character input method can be devised which isbased only on swishes (as opposed to other techniques which rely on morecomplex strokes and/or a combination of swishes with taps and/or othercomplex strokes. In order to use swishes for text entry, a designprocess can follow the following basic procedure. First, define a typeof stroke, (e.g., the swish stroke), that is simple but specific enoughto be distinguishable from any other strokes produced by a touchingdevice during any possible actions that are not related to text input.Second, map every symbol or character to be entered (i.e. the keyboard)to this type of stroke (swish). In this way, the set of actions forentering text is disjoint from all other actions involving the touchingdevice. This allows the text entry area to occupy as much of thedisplay's real estate for the text input method as desired, up to andincluding the entire display. Additionally, in certain embodiments, itmay be unnecessary to use a special stroke to switch between anapplication mode and a character entry mode.

One exemplary mapping from swishes to symbols is represented by an X-Ygrid 50 as exemplified in FIG. 3. In this layout 4–5 characters arerepresented by each cell (e.g., cells 51 and 53) in the grid. In thisexample layout, no attempt has been made to optimize the placement ofthe characters, but such optimization and rearrangement of the locationand content of the characters in the grid is contemplated in certainembodiments of the present invention. Thus, many other layouts can beused without departing from the present invention.

In this layout, the character position in the grid represents adirection of movement used to enter a particular character. Thus, thecharacter “3” corresponds to an upward movement of the swish, while thecharacter “G” represents a right to left swish. A swish used to select aparticular character starts in the cell containing the character(anywhere in the cell) and a swish stroke is carried out in thedirection dictated by the character placement. Thus, a unique point(e.g. the touching point, or some average point along the stroke) aswell as a direction is associated with each stroke. The cell of the gridcontaining the point associated with a stroke together with thedirection uniquely determines the symbol entered.

Using the same grid as illustrated in FIG. 3, FIG. 4 depicts severalexemplary swish strokes that can be used to generate a character input.The arrows start at the touching point corresponding to point “a” inFIG. 1 and FIG. 2, the arrow head indicates the releasing pointcorresponding to point “b” in FIG. 1 and FIG. 2. Thus, in thisillustrative example, stroke 52 is used to input the character “Z”,stroke 54 is used to input the character “J”; stroke 56 is used to inputthe character “Q”; and stroke 58 is used to input the character “A”. Inthis exemplary embodiment, each of the characters represented in thegrid can be similarly input into a device using only swish strokes.

Considering stroke 56 by way of example and not limitation, this strokecan be characterized by capturing the x-y location of the initial touchpoint 59 and a time associated therewith, the x-y location of anintermediate touch point 60 along the path of the stroke and a timeassociated therewith, and the x-y location of a release point 61 and atime associated therewith. These points and times can be used in themanner described later to characterize whether or not the stroke can beclassified as a swish stroke or not, and further to determine a mappingto a character represented by the swish stroke.

In order to distinguish between the entry of text characters asillustrated above, and other commands for any given command, taps (ortaps in combination with other strokes, or other strokes) can be usedfor commands. A grid such as that illustrated in FIG. 3 can be displayedas an overlay (or underlay, or in different colors, differentintensity—fixed or variable, etc.) to a display together with windows orother application objects on the touch sensitive area. The data inputarea can then be extended up to and including the entire size of thedisplay if desired without conflict or confusion with applicationcommands involving selection of display objects. Such a display isdepicted in FIG. 5 in an environment such as a wireless telephonedevice, messaging or paging device, or PDA.

In this illustrative example, the text entry area represented by thegrid co-occupies substantially all of the display's active area with anapplication's user interface. In certain embodiments, the grid is alwayson simultaneously with the application's user interface. (In otherembodiments, the grid can be reduced in prominence or turned offselectively by the user.) By making the text entry area the maximum size(for small, hand-held devices) no special stylus is required to entertext. Instead, a human finger can provide strokes that form the input.In this example, an email application is illustrated, but any suitableapplication can operate in the same manner. In this example, when a tapoccurs in at a menu selection “button” such as the “Cancel” or “OK”buttons 62 and 64 respectively, the device interprets a tap (aconvenient and common example of a non-swish stroke, but should not beconsidered limiting) to mean that such soft coded buttons are beingactuated. Similarly, a user can tap the “To:” line 70, the “Subject:”line 72 or the “Message” line 74 to initiate an entry into one of theirassociated text entry boxes.

Icons such as the telephone icon 76 or envelope icon 78 can similarly beactivated by a tap of the display in the location of such icons, or byanother suitable non-swish type stroke. If, for example, the “Message:”line 74 is selected, the text entry box associated therewith is openedand the user can proceed by use of swish strokes (or equivalent strokedefined to provide an unique character entry stroke type) to enter themessage of an email, in this example. Since the type of stroke used isinherently different for swishes and taps, the device can readilydetermine that an application selection is being made versus a characterentry. Thus, the set of text input commands is disjoint from any othercommand of all possible application commands, so that the text inputtask can be always be active without ever interfering with or creatingambiguous input for other applications (including the operating system).This also permits the text entry screen to simultaneously occupy alarger amount of the display's real estate, up to and including theentire display.

Once a user becomes proficient at text entry, the user may be able toenter text without need for a character overlay. Or, the overlay may besimplified (e.g., by use of only lines dividing the screen into cells,or by using a more faint overlay that is less intrusive to the displayof application display information). In certain embodiments, the overlaycan be selectable to be a solid grid with characters that appear ratherboldly on the display overlaying the application interface (e.g., email,contact list, calendar, text processing, graphics, mapping, games,spreadsheet, dictionary, encyclopedia, reference material, puzzles,etc.). In this mode, the grid and associated characters (collectivelythe “grid”) is always prominently displayed overlaying the applicationinterface. Once the user becomes comfortable with the location of thecharacters within the grid, the characters can fade in brightness orcontrast or change in color to become less prominently displayed on thescreen and can ultimately be turned off to eliminate the characters.Similarly, the grid lines themselves can be reduced in intensity ofbrightness, contrast, color etc. to become less prominent. Additionally,the grid can change from solid lines to broken or dashed lines, may bereduced in prominence to + markings at the intersections and/or singledashes at the periphery of the display or use other techniques to makethem less prominent if desired for the more experienced user. Thus, thegrid and associated characters or commands (collectively “the grid”) canbe displayed in multiple modes wherein the prominence of the grid isvaried in the different modes to accommodate the user's preference andto facilitate both inexperienced and experienced users. Other variationswill occur to those skilled in the art upon consideration of the presentteaching.

Thus, the grid can form a part of the user interface, wherein a featureof the grid, such as the characters or the grid lines for example, isdisplayed prominently in a first display mode and less prominently in asecond display mode. The grid feature is displayed in the second displaymode in a different manner than in the first display mode by use of adifferent color, a lesser intensity, a lower contrast, dashed lines,broken lines, etc.

FIG. 6 depicts one simple process 100 that such a hand-held device suchas a telephone, messaging or paging device, PDA, etc., can use tooperate using the above described swish-based text entry system. In thisprocess, the only possible inputs are swish strokes and taps, and thesystem uses swishes exclusively for text entry and, by way of example,uses tap strokes (or other non-swish strokes) for application specificselections. In other embodiments, other unique stroke classificationscould be equivalently substituted or added. Additionally, more than twostroke type classifications (e.g., swish, non-swish type 1, non-swishtype 2, etc.) could be used without departing from the invention.

The process begins at 104 after which the device awaits a stroke (eithera swish or a non-swish stroke such as a tap) of the touch sensitivedisplay. If no stroke is detected, the process waits at 112 until astroke is detected at 108. At this point, an interrupt may be generatedby the display interface circuitry or other action taken by the device'sprocessor as a result of detection of the stroke and control passes to116 where a determination is made to distinguish whether a swish or anon-swish stroke such as the exemplary tap has been entered, andcategorize it as such. This determination can be made for the case ofswish versus non-swish in certain embodiments, by detecting a movementof the touch location and determining at least one touch location andtouch time associated with the movement; detecting a release locationand time; and categorizing the stroke as a swish type stroke if: 1) thedistance between the location of the release location and the initialtouch location is larger than a distance threshold, 2) the initial touchlocation is within one of a set of specified regions of the touchsensitive medium, and 3) the difference in time between the touch timeand the release time is less than a time threshold (e.g., 150–300 ms,for example. Other thresholds may also be suitable in certainembodiments).

In addition, the process may involve associating the stroke with one ofa plurality of regions on the touch sensitive medium defined by at leastone of the initial touch location (e.g., 59), the initial touch time,the intermediate touch location (e.g., 60), the intermediate touch time,the release location (e.g., 61) and the release time; associating theswish operation with one of a plurality of types defined by at least oneof the initial touch location, the initial touch time, the intermediatetouch location, the intermediate touch time, the release location andthe release time; and conducting an action defined for the associatedtype of swish operation at the associated region. In the case of a swishstroke (according to the most stringent definition), the stroke is asimple stroke wherein the stroke moves in an approximately straight linebetween the initial touch location and the release location. Thus, theintermediate touch location should be approximately along the linedefined by the touch location and the release location.

If a swish was entered, the process determines the grid cell containingthe swish's touch point at 120. The process also determines thedirection from the touch point to the release point at 124. These twopieces of information (which may be gathered in any desired orderwithout departing from the invention) can then be used to map the swishto a desired input character at 128 and the character is selected as thecharacter that the user desires to enter. Control then returns to 108 toawait the next stroke.

Thus, in certain embodiments consistent with the present invention, auser interface method for an electronic device involves detecting astroke of a touch sensitive medium forming a part of the electronicdevice, the stroke occurring on a region of the touch sensitive mediumthat could represent either one of an application function and acharacter input function; determining if the stroke can be categorizedas a first type of stroke, wherein the first type of stroke correspondsto character input stokes, wherein the determination is based at leastupon touch location, release location, and contact time associated withthe stroke; if so, converting the stroke to a character input function;if not, categorizing the stroke as a second type, and translating thestroke to the application function.

In another embodiment consistent with certain embodiments of the presentinvention, a user interface method for an electronic device involvesdetecting a stroke of a touch sensitive medium forming a part of theelectronic device, the stroke occurring on a region of the touchsensitive medium that could represent either one of an applicationfunction and a character input function; determining if the stroke canbe categorized as a first type of stroke, wherein the first type ofstroke corresponds to a character input stoke, and wherein the firsttype of stroke is used to represent each available character that can beinput and is not used in any application function; if so, converting thestroke to a character input function; if not, categorizing the stroke asa second type, and translating the stroke to the application function.

In yet another embodiment consistent with certain embodiments of theinvention, a user interface method for an electronic device involvesdetecting a stroke of a touch sensitive medium forming a part of theelectronic device; categorizing the stroke as one of a swish type strokeand a non-swish type stroke; if the stroke is a swish type stroke,converting the swish type stroke to a character input function; if thestroke is a non-swish type stroke, translating the stroke into anapplication function.

The processes previously described can be carried out on a programmedgeneral-purpose computer forming a part of a hand-held electronic devicesuch as a PDA, messaging or paging device, or a wireless or wiredtelephone. Such as the exemplary computer 150 is depicted in FIG. 7.Computer 150 has a central processor unit (CPU) 154 with an associatedbus 158 used to connect the central processor unit 154 to Random AccessMemory 162 and/or Non-Volatile Memory 166 in a known manner. A touchsensitive display that forms at least a part of the input/outputmechanism at 170 is provided as described above to provide a userinterface. Computer 150 also may have disc storage 174 or othernon-volatile mass storage for storing large amounts of informationincluding, but not limited to, program files and data files. Computer150 may be coupled to a local area network (LAN) and/or wide areanetwork (WAN) and/or the Internet using a network interface 180 such asan Ethernet interface or a wireless LAN interface. In the case of a PDA,paging or messaging device or wireless telephone device, such networkinterface 180 also has wireless transmitter and receiver circuitry,which is not shown in detail to retain the generic nature of thisfigure. Those skilled in the art will appreciate, after consideration ofthe present teachings, that the computer 150 is similar or identical instructure to the computer structure that already forms a part ofconventional PDAs, wireless telephones, messaging and paging devices,and the like; and can be adapted to carry out the process describedabove by a revision of the programming of the existing computer; or aseparate computer can be provided without departing from the presentinvention.

Thus, when implemented using a programmable processor, a computerreadable storage medium can contain instructions that, when executed onthe programmed processor carries out a user interface method for anelectronic device, the instructions including: a segment of code thatdetects a stroke of an area of a touch sensitive medium that couldrepresent either an application function or a character input function,the touch sensitive medium forming a part of the electronic device; asegment of code that categorizes the stroke as one of a first type and asecond type, wherein the second type of stroke is used to represent eachavailable character that can be input and is not used in any applicationfunction; a segment of code that branches to a first operation if thestroke is of the first type and to a second operation if the stroke isof the second type; the first operation being carried out by a segmentof code that translates the stroke to an application function; and thesecond operation being carried out by a segment of code that convertsthe stroke to a character input function.

Thus, an electronic apparatus such as a PDA, wireless telephone, pagingdevice or messaging device, consistent with certain embodiments of thepresent invention has a touch sensitive display that provides amechanism of user output and input. A computer receives input from thetouch sensitive display and provides an output thereto. A user interfacecomputer program runs on the computer, and carries out a process of: adetecting a stroke of a touch sensitive display forming a part of theelectronic device; categorizing the stroke as one of a first type and asecond type. If the stroke is of the first type, the user interfacecomputer program translates the stroke to an application function, andthe computer carries out the application function. If the stroke is ofthe second type, the user interface computer program maps the stroke toa character, and the character is entered as data.

Another embodiment of a an electronic apparatus such as a PDA, wirelesstelephone, paging device or messaging device, consistent with certainembodiments of the present invention has a touch sensitive display orother touch sensitive medium that provides a mechanism of user outputand input; a computer receiving input from the touch sensitive displayand providing an output thereto; a user interface program running on thecomputer, that carries out a process of: detecting a stroke of a touchsensitive display forming a part of the electronic device; categorizingthe stroke as one of a swish type stroke and a non-swish type; if thestroke is of the non-swish type, translating the stroke to anapplication function; and if the stroke is of the second type,converting the stroke to a character input function.

It is beyond the scope of this discussion to define the best layout of atext entry grid for swish-based text input. However, it is noted thatthere is probably no single layout that would be preferred by all users.Some users are already familiar with the mapping on phone keypads (2abc,3def, etc.) while others are not. Some users don't mind a learning phaseif it pays off later in terms of speed and they would enjoy an optimizedlayout, while others want the layout to be intuitive and easy tomemorize. Others may wish to conform to an alphabet based, QWERTY,style, or other layout. Other layouts may be based upon geometries otherthan a grid of cells. Each of these potential realizations iscontemplated, as is an implementation wherein the user is able to selecta layout from a plurality of different layouts.

Alphabets with large numbers of symbols, as well as caps and lower casecharacters, can potentially be addressed in several different ways. Forexample, with out intending to impose any limitation, there can be morethan one layout (for lower case and upper case characters, for digits,for punctuation and special symbols), and switching from one to theother can be accomplished either by a swish, a unique gesture, or otherselection mechanism. In other embodiments, a single layout containingall possible symbols can be used. If large areas of display real estateare used, the number of symbols can be quite high by simply increasingthe number of cells (or other zones) of the grid, or by increasing thenumber of discriminating directions in each cell or zone. Alternatively,it might be easier to relax the definition of swish by loosening one ofthe constraints D, E, or F, while keeping the set of admissible swishesdisjoint from the set of commands of all applications. In an alternativeembodiment, two types of swish strokes can be permitted. In such anembodiment, allowing either slow swishes, or swishes of medium length,or swishes going back and forth, for example, permits the mapping ofpairs of symbols (e.g. lower case/upper case, opening/closing bracket,etc.) in a mnemonic way into pairs of basic/complex swishes. Thus,uppercase and lowercase swishes can be distinguished without shiftingfrom a lowercase mode to an uppercase mode. Other variations will occurto those skilled in the art upon consideration of the present teachings.

In one exemplary embodiment, alphabets with a large set of symbols canbe handled by decomposing the set into several subsets of symbols, theunion of which is the initial set and which can possibly have non-emptyintersection. The symbols of each subset can be arranged into differentlayouts, with only one layout being shown at a time on the displayoverlay. One of these layouts can represent the main layout and is theone that is shown by default. The other layouts are auxiliary layouts.In the main layout, there can be one special text input command for eachauxiliary layout. A swish stroke can then be used to switch the activelayout from the main layout to the auxiliary layout. This switch can beof two types depending on the auxiliary layout: either permanent, (i.e.,until the next change) or temporary, (i.e. valid for the next text inputcommand only). If the two types of switches are desired for oneparticular auxiliary layout, then two different special text inputcommands should be available from the main layout, one to switch it.Each auxiliary layout allowing permanent switch should contain onespecial text input command in the form of a swish to get back to themain layout.

For example, an auxiliary layout containing digits (plus eventuallyother related symbols such as +, −, *, /, =, (, ), #, etc.) might beswitched to in permanent mode only, since in most cases, when there isneed to enter digits, a sequence of them is actually entered. On theother hand, it is desirable to be able to switch to a layout containingthe upper case letters in two different ways: permanently when we needto enter an all caps word, or temporarily, when we need to enter acapitalized word. A layout containing special symbols such as @, $, %,&, ˜, and accentuated letters, could for example be switched to onlytemporarily.

In certain embodiments consistent with the present invention, a swishkeypad can reduce the number of keys by increasing the set of strokesallowed on each key. The same idea can be exploited further for anelegant solution to decrease the number of layouts, possibly to onesingle main layout, by recognizing different types of swishes. Everystroke can then be categorized into more than two groups: non swishstrokes (as before, send to the application), simple swish strokes,complex swish strokes, possibly allowing for different types of complexswish strokes. Two types of swishes, simple and complex (or more), canbe defined for example as follows: short and long strokes;unidirectional strokes and go-and-return strokes (starting like anunidirectional stroke and coming back); or fast and slow strokes. Eachpair of swishes, simple and complex, can be mapped to a pair oflogically related symbols of the alphabet such as: lower/upper caseletters, open/close parenthesis (( ), [ ], { }, < >), single/doublequote, dot/colon, comma/semi-colon, slash/backslash, and so on.

By appropriate definition of the concept of a swish stroke, which is asimple location dependent stroke, this stroke is specific enough to makeit distinct from any strokes produced by “mouse-like” actions, such astap, select, drag, scroll, etc. All possible symbols that arepotentially entered as text on a touch-sensitive device can be thusmapped to swishes. By such a definition, the location of swishes canoccupy large portions of the display real estate, up to and includingsubstantially the entire active region of the display. A swish can beproduced by finger input even on a very small device (i.e., no stylus isnecessary—but could optionally be used). This swish based text entry isfast, accurate and can be silent. Text input based on swishes can beconsistent throughout all applications and can be permanently availableon the device without introducing potential confusion as to characterentry and application commands. The user can potentially choose his/herfavorite layout among a variety of proposed layouts, and an expert usermay be able to input text without looking at the display.

Thus, a user interface method and apparatus for an electronic deviceoperates in accordance with certain embodiments consistent with thepresent invention by detecting a stroke of a touch sensitive displayforming a part of the electronic device. The stroke is categorized asone of a tap stroke or other non-swish stroke and a swish stroke (orother appropriate categories). If the stroke is a tap stroke, it istranslated into an application function. If the stroke is a swishstroke, it is mapped to a character. The touch sensitive display has agrid containing cells, with each cell containing a plurality ofcharacters, and the grid can overlay application interface to therebyoccupying a common area, up to and including the entire area, of thetouch sensitive display. By virtue of definition of the swish stroke asa stroke that can only be used for character input functions, devicesconsistent with certain embodiments of the present invention can operatefull time in a mode that behaves as a text entry mode without need touse special commands or strokes to switch between modes, and eliminatesor reduces the ambiguity between text entry and application functions.

While embodiments of the present invention have been described inconjunction with character input applications, certain embodimentsconsistent with aspects of the present invention can be devised in whichcommands and other functions that are repetitively used with high enoughfrequency within a particular application or group of applications canbe dealt with as a character or collection of characters as describedabove. In one example, an overlay can be devised for an action game orcollection of such games for use to control character movement or otheractions. Such commands can be presented in an overlay in the same manneras with the character set examples above. In addition, other commoncommands can similarly be categorized along with characters orcollections thereof to implement common functions. Those skilled in theart will appreciate that in this context, such commands behave exactlyas if they are a part of the character set and can thus be consideredthe same as a character.

Thus, an input/output apparatus for an electronic device, can beprovided in which a touch sensitive display having a touch sensitivedisplay area has a grid overlaying a portion of the display area, thegrid representing an arrangement of commands that can be entered. Anapplication interface is displayed on the same portion of the displayarea simultaneously with the grid, wherein the electronic deviceresponds to a predetermined type of touch stroke to enter commands fromthe arrangement of commands on the grid into the application. Commandscan thereby be entered, for example, using a predetermined type of touchstroke such as a swish stroke. The grid can be displayed prominently ina first display mode and less prominently in a second display mode,wherein the grid is displayed in the second display mode in a differentmanner by use of one or more of a different color, a lesser intensity, alower contrast, dashed lines, broken lines, etc.

Those skilled in the art will recognize that the present invention hasbeen described in terms of exemplary embodiments based upon use of aprogrammed processor. However, the invention should not be so limited,since the present invention could be implemented using hardwarecomponent equivalents such as special purpose hardware and/or dedicatedprocessors, which are equivalents to the invention as described andclaimed. Similarly, general purpose computers, microprocessor basedcomputers, micro-controllers, optical computers, analog computers,programmable logic circuits, dedicated processors and/or dedicated hardwired logic may be used to construct alternative equivalent embodimentsof the present invention.

Those skilled in the art will appreciate that the program steps andassociated data used to implement the embodiments described above can beimplemented using any suitable computer readable storage medium such asfor example disc storage, Read Only Memory (ROM) devices, Random AccessMemory (RAM) devices, semiconductor storage elements, optical storageelements, magnetic storage elements, magneto-optical storage elements,flash memory, core memory and/or other equivalent storage technologieswithout departing from the present invention. Such alternative storagedevices should be considered equivalents.

The present invention, as described in embodiments herein, has beendescribed as being implemented using a programmed processor executingprogramming instructions that are broadly described above in flow chartform that can be stored on any suitable computer readable storage medium(e.g., disc storage, optical storage, semiconductor storage, etc.) ortransmitted over any suitable electronic communication medium. However,those skilled in the art will appreciate that the processes describedabove can be implemented in any number of variations and in manysuitable programming languages without departing from the presentinvention. For example, the order of certain operations carried out canoften be varied, additional operations can be added or operations can bedeleted without departing from the invention. Error trapping can beadded and/or enhanced and variations can be made in user interface, textentry grid, and information presentation without departing from thepresent invention. Although an email application has been used as anillustrative application, the present invention can be used inconjunction with any suitable application and is clearly not limited toonly the example email application illustrated. Such variations arecontemplated and considered equivalent.

While the invention has been described in conjunction with specificembodiments, it is evident that many alternatives, modifications,permutations and variations will become apparent to those of ordinaryskill in the art in light of the foregoing description. Accordingly, itis intended that the present invention embrace all such alternatives,modifications and variations as fall within the scope of the appendedclaims.

1. A user interface method for an electronic device, comprising:detecting a stroke of a touch sensitive medium forming a part of theelectronic device, the stroke occurring on a region of the touchsensitive medium that could represent either one of an applicationfunction and a character input function; determining if the stroke canbe categorized as a first type of stroke, wherein the first type ofstroke corresponds to character input stokes, wherein the determinationis based at least upon touch location, release location, and contacttime associated with the stroke; if so, converting the stroke to acharacter input function; and if not, categorizing the stroke as asecond type, and translating the stroke to the application function,wherein the touch sensitive medium comprises a touch sensitive display,and wherein the electronic device further comprises an applicationinterface, and wherein the touch sensitive display has a grid containingcells, with each cell containing a plurality of characters, and whereinthe grid overlays the application interface thereby occupying a commonarea of the touch sensitive display, and wherein the grid is displayedsimultaneously with the application interface, and wherein a feature ofthe grid is displayed prominently in a first display mode and lessprominently in a second display mode.
 2. The user interface methodaccording to claim 1, wherein the feature of the grid is displayed inthe second display mode in a different manner than in the first displaymode and wherein the different manner comprises at least one of adifferent color, a lesser intensity, a lower contrast, dashed lines, andbroken lines.
 3. A user interface method for an electronic device,comprising: detecting a stroke of a touch sensitive medium forming apart of the electronic device; categorizing the stroke as one of a swishtype stroke and a non-swish type stroke; if the stroke is a swish typestroke, converting the swish type stroke to a character input function;and if the stroke is a non-swish type stroke, translating the strokeinto an application function, wherein the touch sensitive mediumcomprises a touch sensitive display, and wherein the touch sensitivedisplay has a grid containing cells, with each cell being associatedwith a plurality of characters; and wherein the converting comprisesidentifying a grid cell of the touch sensitive display containing atouch point of the swish stroke, determining a direction of movementassociated with the swish stroke, and mapping the region and directionto the character, and wherein the electronic device further comprises anapplication interface, and wherein the grid overlays the applicationinterface thereby occupying a common area of the touch sensitivedisplay, and wherein the grid is displayed simultaneously with theapplication interface, and wherein a feature of the grid is displayedprominently in a first display mode and less prominently in a seconddisplay mode.
 4. The user interface method according to claim 3, whereinthe feature of the grid is displayed in the second display mode in adifferent manner than in the first display mode and wherein thedifferent manner comprises at least one of a different color, a lesserintensity, a lower contrast, dashed lines, and broken lines.
 5. Anelectronic apparatus, comprising: a touch sensitive display thatprovides a mechanism of user output and input; a computer receivinginput from the touch sensitive display and providing an output thereto;a user interface program running on the computer, that carries out aprocess of detecting a stroke of a touch sensitive display forming apart of the electronic device, categorizing the stroke as one of a swishtype stroke and a non-swish type, if the stroke is of the non-swishtype, translating the stroke to an application function and if thestroke is of the second type, converting the stroke to a character inputfunction; and a computer program running on the computer, wherein thecomputer program further comprises an application interface, and whereinthe touch sensitive display has a grid containing cells, with each cellcontaining a plurality of characters, and wherein the grid overlays theapplication interface thereby occupying a common area of the touchsensitive display therewith, and wherein the grid is displayedsimultaneously with the application interface, and wherein a feature ofthe grid is displayed prominently in a first display mode and lessprominently in a second display mode.
 6. The electronic apparatusaccording to claim 5, wherein the feature of the grid is displayed inthe second display mode in a different manner than in the first displaymode and wherein the different manner comprises at least one of adifferent color, a lesser intensity, a lower contrast, dashed lines, andbroken lines.
 7. An electronic apparatus, comprising, comprising: atouch sensitive display that provides a mechanism of user output andinput; a computer receiving input from the touch sensitive display andproviding an output thereto; means for detecting a stroke of the touchsensitive display; means for categorizing the stroke as one of a swishtype stroke and a non-swish type stroke; means for translating the tapinto an application function, if the stroke is a non-swish type stroke;and means for converting the swish-type stroke to a character inputfunction, if the stroke is a swish type stroke, wherein the gridoverlays substantially the entire touch sensitive display, and whereinthe grid is displayed simultaneously with the application interface, andwherein a feature of the grid is displayed prominently in a firstdisplay mode and less prominently in a second display mode.
 8. Theelectronic apparatus according to claim 7, wherein the feature of thegrid is displayed in the second display mode in a different manner thanin the first display mode and wherein the different manner comprises atleast one of a different color, a lesser intensity, a lower contrast,dashed lines, and broken lines.
 9. An input/output apparatus for anelectronic device, comprising: a touch sensitive display having a touchsensitive display area; a grid overlaying a portion of the display area,the grid representing an arrangement of characters that can be entered;and an application interface, displayed on the same portion of thedisplay area, wherein the electronic device responds to a first type oftouch stroke to enter characters, and responds to a second type of touchstroke to interact with the application interface, and wherein the firsttype of touch stroke comprises a tap stroke and the second type of touchstroke comprises a swish stroke, and wherein the grid is displayedsimultaneously with the application interface, and wherein a feature ofthe grid is displayed prominently in a first display mode and lessprominently in a second display mode.
 10. The input/output apparatusaccording to claim 9, wherein the feature of the grid is displayed inthe second display mode in a different manner than in the first displaymode and wherein the different manner comprises at least one of adifferent color, a lesser intensity, a lower contrast, dashed lines, andbroken lines.
 11. A user interface method for an electronic device,comprising: detecting a stroke of a touch sensitive medium forming apart of the electronic device, the stroke occurring on a region of thetouch sensitive medium that could represent either one of an applicationfunction and a character input function; determining if the stroke canbe categorized as a first type of stroke, wherein the first type ofstroke corresponds to a character input stoke, and wherein the firsttype of stroke is used to represent each available character that can beinput and is not used in any application function; if so, converting thestroke to a character input function; if not, categorizing the stroke asa second type, and translating the stroke to the application function,wherein the touch sensitive medium comprises a display area and whereina grid overlays a portion of the display area, the grid representing anarrangement of characters that can be entered, and wherein the grid isdisplayed simultaneously with an application interface, and wherein afeature of the grid is displayed prominently in a first display mode andless prominently in a second display mode.
 12. The user interface methodaccording to claim 11, wherein the feature of the grid is displayed inthe second display mode in a different manner than in the first displaymode and wherein the different manner comprises at least one of adifferent color, a lesser intensity, a lower contrast, dashed lines, andbroken lines.
 13. An input/output apparatus for an electronic device,comprising: a touch sensitive display having a touch sensitive displayarea; a grid overlaying a portion of the display area, the gridrepresenting an arrangement of commands that can be entered; and anapplication interface, displayed on the same portion of the display areasimultaneously with the grid, wherein the electronic device responds toa predetermined type of touch stroke to enter commands from thearrangement of commands on the grid into the wherein a feature of thegrid is displayed prominently in a first display mode and lessprominently in a second display mode.
 14. The input/output apparatusaccording to claim 13, wherein the feature of the grid is displayed inthe second display mode in a different manner than in the first displaymode and wherein the different manner comprises at least one of adifferent color, a lesser intensity, a lower contrast, dashed lines, andbroken lines.